|
|
Greetings!
Training Wheels
How can you tell if a training program has accomplished its goals? In this edition of KB CommEntary, we continue with the final installment in our series on designing, implementing, and measuring the effectiveness of training programs. Then the training wheels can come off!
|
|
What's Up at KB COMM?
At KB COMM, we provide mission-critical communication and learning services to clients in a variety of industries. Thanks to our established and new clients for keeping us busy. Recent and ongoing projects include:
- Creating technical documentation for a financial-transaction system developer
- Revising technical documenation for a system software developer
- Creating and revising job descriptions for the Careers website of a high-tech manufacturer
- Writing and editing numerous information products for a healthcare diagnostics manufacturer, including communications to employees from the CEO and executive leadership team, customer letters, trade show presentations and materials, and product marketing brochures
- Summarizing journal articles for inclusion in marketing material for a bioscience company
- Writing, formatting, and performing quality control reviews of large, complex chemical residue studies intended for regulatory submission on behalf of a federally funded, cooperative research organization
- Writing supplements and formatting modules of electronic submissions to FDA for a pharmaceutical manufacturing company
- Reviewing and editing for consistency and accuracy response documents and associated modules for electronic submission to FDA on behalf of a medical device manufacturing company
KB COMM LLC is a certified Women's Business Enterprise. We are officially qualified to participate in corporations' woman- and minority-owned vendor programs.
Please keep us in mind for your communication or training projects. |
|
|
Tips and Tales
Training for Success Part 3: Measuring Success
So you've invested time and money in a training program for employees, and the training has been delivered through one or more of the mechanisms we discussed in July's issue. Now what? How do you know if it worked? Even more important, was it worth the investment?
Perhaps the best-known evaluation methodology for judging training programs is Donald Kirkpatrick's Four Level Evaluation Model, first published in a series of articles in 1959 in the Journal of American Society of Training Directors:
- Reaction----how the learners react to the learning process
- Learning----the extent to which the learners gain knowledge and skill
- Behavior/performance----capability of learners to perform the learned skills while on the job
- Results----the effects on the business or environment resulting from the learners' performance
Level 1. Reaction
This level measures one thing: the learner's perception of (reaction to) the course. You know those questionnaires that are passed out after most training classes? The ones that most people don't even fill out? Yeah, that's what we're talking about here.
This information is certainly useful feedback for the instructional designer and instructor (if any). But in study after study, the evidence shows very little correlation between Level 1 evaluations and how well people actually perform when they return to their jobs. Level 1 is also typically the only evaluation that occurs for training programs. It's unusual that companies even make it to Level 2.
Level 2. Learning
Level 2 evaluation answers the question: Did the learner learn anything? It measures the extent to which participants change attitudes, improve knowledge, and increase skill as a result of participating in the learning process. Usually the evaluation consists of a pre-test administered before the training and a post-test administered afterwards. Compare the two and voila! You have a snapshot of what the learners were able to remember immediately after taking the course. Basically, Level 2 measures retention, which fades quickly if learning is not reinforced immediately on the job. Which leads us to Level 3. . .
Level 3. Behavior/performance
This evaluation involves testing the students' capabilities to perform learned skills while on the job, rather than in the classroom. It determines if the desired performance is now occurring by answering the question: Do people use their newly acquired learnings on the job?
Level 3 evaluation can be performed formally (by testing) or informally (through observation). Since Level 3 evaluation must take place after the learners have returned to their jobs, the actual Level 3 measurements will typically involve someone closely involved with the learner, such as a supervisor.
Most would probably agree that, while Level 1 and 2 evaluations are fine in a limited kind of way, Level 3 is where we finally start to see some real-world evidence of how effective a training program really is. We all would also probably agree that we've never personally been involved in a Level 3 evaluation directly connected to a training program. In our experience, it almost never happens in the corporate world. Not good. . .
Level 4. Results
This final evaluation of the training program measures the program's effectiveness: What impact has the training achieved? It's important to view potential impact from a balanced, broad perspective. A training program can have an impact in any or all of the following areas:
- Financial: A measurement, such as an ROI, that shows a monetary return, or a direct impact such as increased output or productivity.
- Customer: Improving an area in which the organization differentiates itself from competitors to attract, retain, and deepen relationships with its targeted customers.
- Internal: Achieving excellence by improving such processes as supply-chain management, production processes, or support processes.
- Innovation and learning: Ensuring the learning package supports a climate for organizational change, innovation, and the growth of individuals.
As we move from Level 1 to Level 4, the evaluation process becomes more difficult and time-consuming; however, the higher levels provide information that is increasingly significant in value. The most frequent type of measurement is Level 1 because it is the easiest to measure, yet it provides the least valuable data. Measuring results that affect the organization is often considerably more difficult, thus it is conducted less frequently, although it yields the most valuable information.
Working backwards . . .
It's hard to believe companies would invest in any type of program for which they are not prepared to measure the on-the-ground results. But, at least in the case of training, it happens all the time.
One way to help ensure that training programs will actually drive the desired business results is to work backwards through the Kirkpatrick model when planning and designing the training. The planning process should therefore identify:
- The desired impact (outcome or result) that will improve the performance of the business
- The level of performance learners must achieve to create the impact
- The knowledge and skills they need to learn in order to perform
- What the learners need to perceive in order to learn (the need to learn)
We hope you will be able to put to use in your enterprise some of the tips and suggestions we have included in our series on training for success. Please contact us for more information about how to design and implement successful, effective training in your environment.
We haven't decided what gems to impart in our next issue, but we'll come up with something. Until then, enjoy the upcoming holidays! |
|
|
|
|
And please remember, when it has to be right, KB COMM is here to help.
Sincerely, |
|
|
